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DIGITAL CAMERA 

FIELD OF THE INVENTION 
[0001] The present invention generally relates to photography and, 
more particularly, to digital cameras that are equipped to determine whether the . 
eyes of a photographed person are open or closed. 

BACKGROUND OF THE INVENTION 
[0002] Photographs of persons with their eyes closed are not desirable. 
However, amateur photographers often take quick snapshots of groups of 
humans (for example, family photographs). Since it is natural for humans to blink 
their eyes, one or more persons in a photograph may have their eyes closed. 
Therefore, it is desirable to provide an improved digital camera that alerts the 
camera operator when one of the persons in a photograph have their eyes 
closed or performs some other corrective action that enables the photographer to 
take better pictures. Conventional camera are not equipped to provides such 
features. It is further envisioned that this camera may be used for personal 
identification applications based on iris recognition technology. 

SUMMARY OF THE INVENTION 
[0003] In accordance with the present invention, an improved digital 
camera is provided that detects pictures of human subjects whose eyes are not 
closed. The improved digital camera generally includes an eye detection 
activation element that is operable to initiate an eye detection mode; an image 
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sensor that captures image data for a target subject; an image processor that 
converts the image data into a digital image form; and an eye detection 
component that evaluates the digital image form for the target subject and 
determines whether at least one eye of the target subject is closed. 

[0004] Further areas of applicability of the present invention will 
become apparent from the detailed description provided hereinafter. It should be 
understood that the detailed description and specific examples, while indicating 
the preferred embodiment of the invention, are intended for purposes of 
illustration only and are not intended to limit the scope of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0005] The present invention will become more fully understood from 
the detailed description and the accompanying drawings, wherein: 

[0006] Figure 1 is a block diagram depicting the principal components 
of an exemplary camera system designed according to the present invention; 

[0007] Figure 2 is a flow chart illustrating the operation of the 
exemplary camera system in accordance with the present invention; and 

[0008] Figure 3 is flow chart illustrating how face detection processing 
may be integrated into the operation of the exemplary camera system in 
accordance with the present invention; and 

[0009] Figure 4 is a flow chart illustrating an exemplary technique for 
digitally enhancing captured image data in accordance with the present 
invention. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0010] Figure 1 illustrates the principal components of an exemplary 
digital camera system 1 0 in accordance with the present invention. The digital 
camera system 10 generally includes a shutter mechanism 12, an image sensor 
14, an image processor 16, a data store, and a user interface 18. The digital 
camera system 10 further includes an eye detection activation element 19, an 
eye detection component 20, and, optionally, a face detection component 22. As 
will be further described below, the data store may be further defined to include a 
transient memory space 24 as well as a permanent memory space 26. It is 
readily understood that only the principle components are discussed herein, but 
that other known camera components may be incorporated into the camera 
system. 

[0011] Operation of the digital camera system 10 is further described in 
relation to Figure 2. To photograph a subject, the camera operator aims the 
camera at the target subject and actuates the shutter mechanism 12 via a push 
button or other user input element. It is readily understood that the camera 
operator may locate the subject using a viewfinder as well as adjust various 
known camera settings prior to taking the photograph. The shutter mechanism 
12 in turn cooperatively operates with the image sensor 14 to capture image data 
including for a target subject as shown at step 34. In one exemplary 
embodiment, the image sensor 14 may be in the form of a charge coupled 
device. The image processor 16 is adapted to retrieve the captured image data 
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from the image sensor. The image processor 1 6 then converts the image data 
into a digital image form and stores the digital image form in the transient 
memory space for additional processing by the camera 10. 

[0012] In accordance with the present invention, the camera operator 
may activate an eye detection feature prior to taking the photograph as shown at 
step 32. For instance, when the camera operator ascertains that the target 
subject has eyes, the camera operator activates the eye detection feature using 
the eye detection activation element. Conversely, if the target subject does not 
include eyes, the camera operator may deactivate, the eye detection feature 
using the eye detection activation element. It is envisioned that the eye detection 
activation element may take a variety of forms, including (but not limited to) push 
buttons, radial knobs, a touch screen display, or a combination thereof. 
Alternatively, digital camera system may be configured so that the eye detection 
feature is performed each time a photograph is taken, 

[0013] In an eye detection mode, the eye detection component 20 is 
operable to evaluate the digital image form at step 36 to determine whether the 
eyes of any faces in the target subject are open or closed. The eye detection 
component 20 may employ one or more suitable algorithms to evaluate the 
digital image form. An exemplary eye detection algorithm is further described in 
an article by Saad A. Sirohey and Azriel Rosenfeld, entitled Eye Detection In A 
Face Image Using Linear and Non-Linear Filters, 36 Pattern Recognition 1 387- 
1391 (2001). However, it is readily understood that other known eye detection 
algorithms are also within the broader scope of the present invention. 
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[0014] When the eyes of the target subject are deemed open, the eye 
detection corDponent 20 nnoves the digital image form from the transient memory 
space to a permanent memory location as shown at step 40. In an alternative 
embodiment, the digital camera system may be employed in personal 
identification applications. To facilitate the identification process, the camera is 
preferably operable to perform iris recognition processing on the digital image 
form only when the eyes of the target subject are deemed open. Iris recognition 
may be implemented using one of a various well known algorithms. 

[0015] On the other hand, when at least one eye of the target subject is 
deemed closed, the eye detection component 20 is further operable to perform at 
least one additional camera function based on this observation. In one 
embodiment, the eye detection component 20 cooperatively operates with the 
user interface to provide an alert to the camera operator as shown at step 42. It 
is readily understood that the alert may be audible, visual, or some other suitable 
form. 

[0016] In addition to merely being alerted, the camera operator may be 
given the option to perform one or more corrective actions in relation to the 
recently taken photographs. For example, the camera operator may view the 
recently taken photograph using a display provided by the camera as shown at 
step 35. When viewing the photographs, the camera operator may determine if 
the photograph meets their needs, or if another photograph should be taken of 
the target subject. Furthermore, the camera operator may be given the option of 
deleting the corresponding digital image from the transient memory space (or to 
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designate the digital image form for subsequent deletion). It is understood that 
the digital camera may be equipped with a view screen or other display device 
(e.g., an LCD display) for viewing the most recently taken photograph (s). 

[0017] In one exemplary embodiment, the camera may be configured 
to support a single picture review mode and/or a multiple picture review mode. In 
a single picture review mode, the camera operator is prompted to individually 
review each photograph immediately after it has been taken. The camera 
operator is then given the option to retain or discard the photograph as shown at 
step 44. If the camera operator designates the photograph as being acceptable, 
then the corresponding digital image form is moved from its transient memory 
space to a permanent data store as shown at 40. Conversely, if the camera 
operator designates the photograph as being unacceptable, the corresponding 
digital image form is deleted from the transient memory space as shown at 46. 
In either case, it is understood that the camera provides suitable user input 
elements to ascertain the operators selection. 

[0018] In a multiple picture review mode, the camera operator initially 
defers reviewing recently taken photographs. Thus, the camera operator may 
shoot a sequence of photographs. At the time the photographs are taken, the 
camera operator is provided an alert for each photograph that is deemed to have 
one or more closed eyes therein. At some later time, the camera operator is 
given the option to take some corrective action with respect to each of the 
deficient photographs. For instance, each recently captured photograph is 
individually displayed to the camera operator. Alternatively, only the defective 
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photographs may be displayed to the camera operator. In either case, the 
camera operator is then given the option to retain or discard the displayed 
photograph. To facilitate review, the photographs may be sorted sequentially in 
time or, alternatively (at operators discretion), may be sorted according to the 
degree that the eyes are deemed to be closed. Once an option has been 
selected, the next photograph is displayed to the camera operator. In this way, 
the camera operator may select and save the preferred photographs from a 
series of photographs that may have been taken over a short period of time. 

[0019] Alternatively, the camera may perform some corrective action 
without intervention by the camera operator. For instance, the camera may 
disregard photographs where the subject's eyes are deemed closed without 
prompting the operator. To decreasing processing time, each image in a 
sequence of images may not be evaluated by the eye detection component. If 
the subject is blinking, then a few milliseconds are generally needed to the 
subject's eyes to fully open. Therefore, when blinking is detected in an image, 
the camera may simple ignore the next few images in the sequence of images 
(e.g., 2-3 images depending on the speed of the camera). In other words, the 
camera does not apply the eye detection algorithm to this image data, thereby 
reducing processing time. 

[0020] In another exemplary embodiment, the camera eliminates the 
subject's closed eyes from the photograph by digitally enhancing the captured 
image data as shown in Figure 4. First, the camera operator provides or 
othenA/ise inputs at step 52 an indicator of the target subject into the camera. 
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The camera operator then proceeds to capture at step 54 one or more images of 
the subject. The captured image data is evaluated at step 56 using an eye 
detection algorithm as further described above. When the eyes of the target are 
deemed open, the eye detection component is operable at step 60 to store an 
indicator of the subject along with corresponding image data in a database. 

[0021] During the same sequence of captured images or at some 
subsequent time, the stored image data may be used to corrective deficient 
images as further described below. When the eyes of the target are deemed 
closed, the camera may apply a substitution process to digitally replace the 
closed eye image data with open eye image data taken from previously stored 
image data for the subject. To do so, the applicable image data is retrieved at 
step 62 from the database. Known blending techniques are then used at step 64 
to incorporate portions of the image data having eyes of the subject into the most 
recently captured image data of the subject. In this way, the camera may 
enhance the captured image data. 

[0022] In another aspect of the present invention, the digital camera 
system 10 may be further equipped with face detection component 22. The face 
detection component cooperatively operates with the eye detection component to 
evaluate the digital image form as shown in Figure 4. Specifically, the face 
detection component identifies one or more faces associated with the target 
subject at step 72 and partitions the digital image form into partitioned data 
segments at step 74, such that each data segment includes at least a portion of a 
single face. To do so, the face detection component employs a suitable face 
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detection algorithm such as the one set forth by Chellappa et ai. in Human and 
Machine Recognition of Faces: A Survey, Proceedings of the IEEE, Vol, 83, 
No. 5 (May 1995). It is readily understood that other known face detection 
algorithms are within the scope of the present invention. 

[0023] Each partitioned data segment then serves as an input to the 
eye detection component. For each partitioned data segment, eye detection 
processing is performed at step 78 as described above. When one eye in the 
face contained in the partitioned data segment is deemed closed, the camera 
proceeds with an additional camera function at step 82. When the eyes in the 
face contained in the partitioned data segment are deemed open, then the next 
partitioned data segment is retrieved for processing by the eye detection 
component as shown at step 76. Once all of the partitioned data segments have 
been analyzed (and It is determined that the eyes in each of the faces contained 
in the target subject are open), the eye detection component 20 will move the 
digital image form to a permanent memory location as shown at step 86. 

[0024] The description of the invention is merely exemplary in nature 
and, thus, variations that do not depart from the gist of the invention are intended 
to be within the scope of the invention. Such variations are not to be regarded as 
a departure from the spirit and scope of the invention. 
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